Gait analysis using a single depth camera

Abstract—Gait analysis is often used as part of the rehabilitation program for post-stoke recovery assessment. Since current optical diagnostic and patient assessment tools tend to be expensive and not portable, this paper proposes a novel marker-based tracking system using a single depth camera which provides a cost-effective solution suitable for home and clinic use. The proposed system can simultaneously generate motion patterns even within a complex background using the proposed geometric model-based algorithm and autonomously provide gait analysis results. The processed rehabilitation data can be accessed by cross-platform mobile devices using cloud-based services enabling emerging telerehabilitation practices. Experimental validation shows a good agreement with state-of-the-art non-portable and expensive industrial standards

Gait phase classification for in-home gait assessment

With growing ageing population, acquiring joint measurements with sufficient accuracy for reliable gait assessment is essential. Additionally, the quality of gait analysis relies heavily on accurate feature selection and classification. Sensor-driven and one-camera optical motion capture systems are becoming increasingly popular in the scientific literature due to their portability and cost-efficacy. In this paper, we propose 12 gait parameters to characterise gait patterns and a novel gait-phase classifier, resulting in comparable classification performance with a state-of-the-art multi-sensor optical motion system. Furthermore, a novel multi-channel time series segmentation method is proposed that maximizes the temporal information of gait parameters improving the final classification success rate after gait event reconstruction. The validation, conducted over 126 experiments on 6 healthy volunteers and 9 stroke patients with handlabelled ground truth gait phases, demonstrates high gait classification accuracy

A depth camera motion analysis framework for tele-rehabilitation : motion capture and person-centric kinematics analysis

With increasing importance given to telerehabilitation, there is a growing need for accurate, low-cost, and portable motion capture systems that do not require specialist assessment venues. This paper proposes a novel framework for motion capture using only a single depth camera, which is portable and cost effective compared to most industry-standard optical systems, without compromising on accuracy. Novel signal processing and computer vision algorithms are proposed to determine motion patterns of interest from infrared and depth data. In order to demonstrate the proposed framework’s suitability for rehabilitation, we developed a gait analysis application that depends on the underlying motion capture sub-system. Each subject’s individual kinematics parameters, which are unique to that subject, are calculated and these are stored for monitoring individual progress of the clinical therapy. Experiments were conducted on 14 different subjects, 5 healthy and 9 stroke survivors. The results show very close agreement of the resulting relevant joint angles with a 12-camera based VICON system, a mean error of at most 1.75% in detecting gait events w.r.t the manually generated ground-truth, and significant performance improvements in terms of accuracy and execution time compared to a previous Kinect-based system

Deep graph regularized learning for binary classification

With growing interest in data-driven classification, deep learning is now prevalent thanks to its ability to learn feature mapping functions solely from data. For very small training sets, however, deep learning, even with traditional regularization techniques, often overfits, resulting in sub-par classification performance. In this paper, we propose a novel binary classifier deep learning method, based on an iterative quadratic programming (QP) formulation with a graph Laplacian regularizer (GLR), combining the merits of model-based and data-driven approaches. Specifically, the proposed network employs a convolutional neural network (CNN) to learn deep features, which are used to define edge weights for a graph to pose a convex QP problem. Further, we design a novel loss function to penalize samples at the class boundary during semi-supervised learning. Results demonstrate that given a small-size training dataset, our network outperforms several state-of-the-art classifiers, including CNN, model-based GLR, and dynamic graph CNN classifiers

Domain knowledge informed multitask learning for landslide induced seismic classification

Automatic seismic signal classification methods are extensively investigated to reduce or replace manual interpretation, with great potential in previous research. Discriminative seismic wave propagation physical characteristics, such as velocities and accelerations, are rarely considered for classification. A multitask learning scheme is proposed that utilises the seismic wave equation and three-dimensional (3D) P-wave velocity Vp model for signal representation learning. The classifier uses the obtained latent feature maps on a convolutional neural network architecture for classification of rockfall, slide quake, earthquake, and natural/anthropogenic noise events, recorded at an ongoing landslide. Our experimental results show that our approach outperforms state-of-the-art methods

Kinematics analysis multimedia system for rehabilitation

Driven by recent advances in information and communications technology, tele-rehabilitation services based on multimedia processing are emerging. Gait analysis is common for many rehabilitation programs, being, for example, periodically performed in the post-stroke recovery assessment. Since current optical diagnostic and patient assessment tools tend to be expensive and not portable, this paper proposes a novel marker-based tracking system using a single depth camera which provides a cost-effective solution that enables tele-rehabilitation services from home and local clinics. The proposed system can simultaneously generate motion patterns even within a complex background using the proposed geometric model-based algorithm and autonomously provide gait analysis results using a customised user-friendly application that facilitates seamless navigation through the captured scene and multi-view video data processing, designed using feedback from practitioners to maximise user experience. The locally processed rehabilitation data can be accessed by cross-platform mobile devices using cloud-based services enabling emerging tele-rehabilitation practices

Distinct feature extraction for video-based gait phase classification

Recent advances in image acquisition and analysis have resulted in disruptive innovation in physical rehabilitation systems facilitating cost-effective, portable, video-based gait assessment. While these inexpensive motion capture systems, suitable for home rehabilitation, do not generally provide accurate kinematics measurements on their own, image processing algorithms ensure gait analysis that is accurate enough for rehabilitation programs. This paper proposes high-accuracy classification of gait phases and muscle actions, using readings from low-cost motion capture systems. First, 12 gait parameters, drawn from the medical literature, are defined to characterize gait patterns. These proposed parameters are then used as input to our proposed multi-channel time-series classification and gait phase reconstruction methods. Proposed methods fully utilize temporal information of gait parameters, thus improving the final classification accuracy. The validation, conducted using 126 experiments, with 6 healthy volunteers and 9 stroke survivors with manually-labelled gait phases, achieves state-of-art classification accuracy of gait phase with lower computational complexity compared to previous solution

Feature selection and extraction in sequence labeling for arrhythmia detection

Automated Electrocardiogram (ECG)-based arrhythmia detection methods replace traditional, manual arrhythmia detection reducing the requirement for trained medical staff. Traditionally, ECG-based arrhythmia detection is performed via QRS complex detection followed by feature extraction, based on hand-crafted features, such as RR-intervals, Fast Fourier Transform-based features, wavelet analysis, higher order statistics and Hermite features. After the features are extracted, the ECG segments are classified into pre-defined categories. This study investigates the value of the feature extraction and selection methods for ECG-based arrhythmia detection. That is, with the emerging trend of deep learning methods which are capable of automatic feature extraction and selection, the research question addressed in this paper is if good classification performance can be obtained by feeding the raw ECG sequence directly into robust classifiers or handcrafted feature extraction/selection is necessary. Classification performance across a range of state-of-the-art classification methods indicates that feeding raw signals into the convolution neural network-based classifiers usually leads to the best performance but at the expense of high inference time

Model selection-inspired coefficients optimization for polynomial-kernel graph learning

Graph learning has been extensively investigated for over a decade, in which the graph structure can be learnt from multiple graph signals (e.g., graphical Lasso) or node features (e.g., graph metric learning). Given partial graph signals, existing node feature-based graph learning approaches learn a pair-wise distance metric with gradient descent, where the number of optimization variables dramatically scale with the node feature size. To address this issue, in this paper, we propose a low-complexity model selection-inspired graph learning (MSGL) method with very few optimization variables independent with feature size, via leveraging on recent advances in graph spectral signal processing (GSP). We achieve this by 1) interpreting a finite-degree polynomial function of the graph Laplacian as a positive-definite precision matrix, 2) formulating a convex optimization problem with variables being the polynomial coefficients, 3) replacing the positive-definite cone constraint for the precision matrix with a set of linear constraints, and 4) solving efficiently the objective using the Frank-Wolfe algorithm. Using binary classification as an application example, our simulation results show that our proposed MSGL method achieves competitive performance with significant speed gains against existing node feature-based graph learning methods

Signal information processing tools for healthcare diagnostics

The smart healthcare monitoring service has been given more attention in recent decades. With rising healthcare demand and progress in image processing, video-based gait assessment becomes a good alternative solution to assess the physical recovery progress for post-stroke survivors. However, most video-based assessment systems, commercially and in the literature, usually requires large laboratory space, are of high cost, and not portable, thus are impractical for in-home use. Accurate, low-cost, portable motion capture systems are growing in popularity, especially those that do not require expert knowledge to operate. This research proposes an alternative single depth camera based OPTIcal Kinematics Analysis system (named 'OPTIKA'). Novel signal processing and computer vision algorithms are proposed to determine motion patterns of interest from infrared and depth data, and enable real-time simultaneous tracking of joints based on attached retroreflective ball markers. Specifically, an accurate trajectory-based gait phase classification system is proposed to facilitate the diagnostics of muscle activities during gait, using readings from low-cost motion capture systems 'OPTIKA'. Feature selection/extraction methods are proposed to enable an automatic segmentation of motion records into individual gait cycles with nine gait phases slice, which provides a more intuitive diagnostics experience for clinical therapists to analyze the rehabilitation progress associated to the kinematics in particular gait periods. This research also analyzes the sensitivity of feature selection/extraction methods against the classification performance in two healthcare monitoring applications. To overcome the limitations of high-cost training data labeling work and when parts of the training labels are noisy, a robust semi-supervised binary classifier is proposed to combine deep learning and graph based signal processing methods. The experiments demonstrate that given an acceptable proportion of noisy training labels, the proposed classifier outperforms several state-of-the-art classifiers. The overall concepts and systems presented in this thesis form an underlying approach for further video-based healthcare monitoring service that assists the diagnostics of physical rehabilitation.The smart healthcare monitoring service has been given more attention in recent decades. With rising healthcare demand and progress in image processing, video-based gait assessment becomes a good alternative solution to assess the physical recovery progress for post-stroke survivors. However, most video-based assessment systems, commercially and in the literature, usually requires large laboratory space, are of high cost, and not portable, thus are impractical for in-home use. Accurate, low-cost, portable motion capture systems are growing in popularity, especially those that do not require expert knowledge to operate. This research proposes an alternative single depth camera based OPTIcal Kinematics Analysis system (named 'OPTIKA'). Novel signal processing and computer vision algorithms are proposed to determine motion patterns of interest from infrared and depth data, and enable real-time simultaneous tracking of joints based on attached retroreflective ball markers. Specifically, an accurate trajectory-based gait phase classification system is proposed to facilitate the diagnostics of muscle activities during gait, using readings from low-cost motion capture systems 'OPTIKA'. Feature selection/extraction methods are proposed to enable an automatic segmentation of motion records into individual gait cycles with nine gait phases slice, which provides a more intuitive diagnostics experience for clinical therapists to analyze the rehabilitation progress associated to the kinematics in particular gait periods. This research also analyzes the sensitivity of feature selection/extraction methods against the classification performance in two healthcare monitoring applications. To overcome the limitations of high-cost training data labeling work and when parts of the training labels are noisy, a robust semi-supervised binary classifier is proposed to combine deep learning and graph based signal processing methods. The experiments demonstrate that given an acceptable proportion of noisy training labels, the proposed classifier outperforms several state-of-the-art classifiers. The overall concepts and systems presented in this thesis form an underlying approach for further video-based healthcare monitoring service that assists the diagnostics of physical rehabilitation